Convergence Acceleration via Chebyshev Step: Plausible Interpretation of Deep-Unfolded Gradient Descent

Deep unfolding is a promising deep-learning technique, whose network architecture based on expanding the recursive structure of existing iterative algorithms. Although convergence acceleration remarkable advantage deep unfolding, its theoretical aspects have not been revealed yet. The first half this study details analysis in deep-unfolded gradient descent (DUGD) trainable parameters are step sizes. We propose plausible interpretation learned step-size DUGD by introducing principle Chebyshev steps derived from polynomials. use (GD) enables us to bound spectral radius matrix governing speed GD, leading tight upper rate. rate GD using shown be asymptotically optimal, although it has no momentum terms. also show that numerically explain well. In second study, %we apply theory and Chebyshev-periodical successive over-relaxation (Chebyshev-PSOR) proposed for accelerating linear/nonlinear fixed-point iterations. Theoretical numerical experiments indicate Chebyshev-PSOR exhibits significantly faster various examples such as Jacobi method proximal methods.